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DETAILED ACTION 



Observation 

1 . Claims 1 , 3, 9, 10, 24, 26, 32, and 33 recite the limitation "optional" or 
"optionally". The limitations recited after the word "optional or "optionally" to claimsl , 3, 
9, 10, 24, 26, 32, and 33 has not been further treated on the merits because "optional" 
is an act of choice. It is not necessary to have those limitations after the word "optional". 

Claim Rejections - 35 USC § 102 

2. The following is a quotation of the appropriate paragraphs of 35 U.S.C. § 1 02 
that form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in 
pubfc use or ™ sale in this country, more than one year prior to the date of appl.cat.on for patent ,n 
the United States. 

3. Claims 1-3, 7-8, 10, 18-19, 22, 24-26, 30-31 , 33, 41-42, and 45 are rejected 
under 35 U.S.C. § 102(b) as being anticipated by Henley, US 5,657,073. 

Regarding claim 1 , Henley discloses an apparatus for creating, managing 
and publishing interactive virtual tours (i.e. panoramic imaging with distortion 
correction and selectable field of view), the apparatus comprising: 

- a panorama data acquisition unit (i.e. a plurality of video cameras 
10) implementing means of capturing panoramic data and 
preparing panoramic data for further processing (see column 2, line 
54 to column 3, line 5); 
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- a transform engine (an image transformation engine 1 6) responsive 
to panoramic data (i.e. data capture from cameras 10) and 
implementing correcting distortions in panoramic data (see column 

4, lines 41 - 48); 

- a viewing engine (i.e. image clipper device 20) responsive to 
panoramic data and implementing perspective correction (see 
column 4, lines 48 - 55); 

- a display (display device 30) for rendering output of viewing engine, 
transform engine, and panoramic data acquisition unit (see column 
4, lines 54-61). 

Regarding claim 2, Henley teaches a panoramic data acquisition unit (i.e. 
a plurality of video cameras 1 0) is adapted to capture data representing 
panoramic images or video (see column 4, lines 22 - 33). 

Regarding claim 3, Henley teaches a panoramic data acquisition unit (i.e. 
a plurality of video cameras 10) comprises at least one versatile device (i.e. the 
multi camera assembly 1 0, a lens assembly 1 1 , and a housing 1 2) for creating 
representations of stimuli covering substantially all directions around a given view 
point comprising at least one grid of one or more sensor elements disposed on 
an N-dimensional and arbitrarily shaped surface (see column 4, lines 22 - 33). 
Since video cameras (10) of Henley is a digital camera (see column 2, lines 13 - 
16), it would have had CCD sensor elements arranged in the two dimensional 
matrix within the shaped surface. 
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Regarding claim 7, Henley teaches a panoramic data acquisition unit 
(video cameras 10) is adapted to capture each complete panoramic data block 
(e.g. block data field, V1) in a single image/video frame (see column 4, lines 33 - 
57). 

Regarding claim 8, Henley teaches a panoramic data acquisition unit (a 
plurality of video cameras 10) is adapted to capture each complete panoramic 
data block in a plurality of image/video frames that are combined to form 
complete panoramic data blocks (V1 - V4, OV12 - OV41) (i.e. a single image of 
the 360 degree panoramic or panospheric field of view collected by the multiple 
cameras; see Figure 2 and also see column 4, lines 33 - 57). 

Regarding claim 10, Henley teaches a panoramic data acquisition unit (a 
plurality of video cameras 10) further comprises of preparing and transferring all 
or portions of acquired panoramic data to transform engine, viewing engine, or 
display (e.g. the data streams 14, as shown entering the image transformation 
engine 16 and exit as a data stream 18, which is directed into an image clipper 
device 20, as shown in Figure 2; see column 4, lines 33 - 57). 

Regarding claim 18, Henley teaches an image transformation engine (16) 
being responsive to user input (joystick) and implements for perspective 
correction of panoramic data (i.e. remove distortions generated by the image 
capturing process; see column 4, lines 34 - 57). 
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Regarding claim 19, Henley discloses the viewing engine (20) further 
implements of navigating panoramic data (i.e. controlled by a pan-tilt-rotation- 
zoom controller means 22; see column 4, lines 34 - 57). 

Regarding claim 22, Henley teaches the representative information 
rendered by control engine (i.e. virtual reality display system) about all or parts of 
virtual tour is rendered in 2D, 3D or higher dimensional space and/or time (i.e. 
three dimensional viewing or stereoscopic imaging; see column 4, lines 58 - 67). 

Regarding claims 24-26, 30-31, 33, 41-42, and 45, these nine claims are 
method claims corresponding to apparatus claims 1-3, 7-8, 10, 18-19, and 22. 
Therefore, method claims 24-26, 30-31, 33, 41-42, and 45 are analyzed and 
rejected as previously discussed with respect to apparatus claims 1-3, 7-8, 10, 
18-19, and 22. 

Claim Rejections - 35 USC § 103 

4. The following is a quotation of 35 U.S.C. § 1 03(a) which forms the basis for all 

obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described 
as set forth in section 1 02 of this title, if the differences between the subject matter sought to 
be patented and the prior art are such that the subject matter as a whole would have been 
obvious at the time the invention was made to a person having ordinary skill in the art to which 
said subject matter pertains. Patentability shall not be negatived by the manner in which the 
invention was made. 

5. Claims 4, 5, 27, and 28 are rejected under 35 U.S.C. § 1 03(a) as being 
unpatentable over Henley, US 5,657,073 in view of Tanida, US 7,009,652. 
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Regarding claims 5 and 28, note the discussion of Henley above. Hanley 
does not mention comprises at least one grid of one or more photosensitive 
elements on a surface with a spherical geometry. In same field of endeavor 
(video camera), Tanida teaches a well-known video camera which has a 
multiplicity of photosensitive pixel elements array on a spherical surface (see 
column 2, lines 22 - 24). It would have been obvious to one of the ordinary skill 
in the art at the time the invention was made to have provided a multiplicity of 
photosensitive elements placed on spherical surface as taught by Tanida to the 
camera or acquisition unit of Henley because the arrangement of photosensitive 
elements on spherical surface of Tanida would obtain a high resolution image 
(see column 2, lines 1 1 - 24 of Tanida). 

As to claims 4 and 27, Tanida clearly teaches sensor elements (i.e. 
photosensitive elements) responsive to electromagnetic radiation (i.e. light beam) 
(see column 4, lines 15-17). 

6. Claim 6 and 29 are rejected under 35 U.S.C. § 103(a) as being unpatentable 

over Henley, US 5,657,073 in view of Keast, US 5,721 ,585. 

Regarding claims 6 and 29, note the discussion of Henley above. Henley 
discloses the field of view of all of the cameras collectively captured is a 
complete 360-degree panoramic field of view. Henley, however, fails to 
specifically disclose a vertical field of view that is usually less than 180 degrees. 
Keast teaches an image system provides both a 180-degree vertical view and a 
360-degree azimuthal view (see column 5, lines 16-28, and column 6, lines 54 - 
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67). It would have been obvious to one of ordinary skill in the art at the time 
invention was made to have added the 180-degree vertical view as taught by 
Keast to the panoramic field of view of Henley because both 180-degree vertical 
field of view and 360-degree panoramic field of view would provide a complete 
and uniform imaging of the sphere of view (column 5, lines 16 - 29). 
7. Claims 9, 14-16, 20-21, 23, 32, 37-39, 43-44, and 46 are rejected under 35 
U.S.C. § 103(a) as being unpatentable over Henley, US 5,657,073 in view of Jackson et 
al., US 5,990,941. 

Regarding claim 9, note the discussion of Henley above. Henley discloses 
an imaging system for the production of virtual tour data includes in either a still 
image or a video format (see column 2, lines 9-11). Henley, however, fails to 
specifically disclose to resources on local and distributed networks. Jackson 
teaches the ability to delivery these image data, still or full motion video via 
broadcast, cable, digital network such as Internet, or on transportable media 
such as CD-ROM, videotape, or laserdisc from a remote site 12 to a local site 14. 
It would have been obvious to one of ordinary skill in the art at the time invention 
was made to have added ability to delivery these image data as taught by 
Jackson to the imaging system of view of Henley because this feature provides 
the spherical image can be transmitted to multiple users at local locations 
(column 10, lines 12 -27). 

Regarding claim 14, Jackson teaches a package generator implement 
specifying active regions on panoramic data. Jackson teaches a so-called "hot 
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spots", as disclosed in column 4, lines 3 - 7, that are identified by a user as the 
user interacts with the stored data files representing the images. 

Regarding claim 1 5, Jackson teaches to specify the navigable paths for 
interactive display of any portion of a spherical image, as disclosed in column 4, 
lines 8 - 10. 

Regarding claim 16, Jackson teaches that the end user could utilize the 
navigable paths or walk-through sequences are navigated in automatic or guided 
mode (i.e. controls the direction of viewing by moving a computer mouse; see 
Figure 9 and also column 8, lines 16-31). 

Regarding claims 20 and 21 , Jackson teaches a viewing engine 
implements means for a single user or a plurality of users to independently and 
simultaneously interact with and navigate panoramic data (see column 8, lines 16 
- 31 and column 10, lines 12 - 27). 

Regarding claim 23, Jackson teaches the view engine and control engine 
comprising a unitary structure (microcomputer and control interface 15 as shown 
in Figure 1). 

Claims 32, 37-39, 43-44, and 46 are method claims. Thus, they are 
analyzed as previously discussed with respect to the apparatus claims 9, 14-16, 
20-21 , and 23 above. 
8. Claims 11-13 and 34-36 are rejected under 35 U.S.C. § 103(a) as being 
unpatentable over Henley, US 5,657,073 in view of Xiong, US 5,960,108. 
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Regarding claim 1 1 , note the discussion of Henley above. Henley teaches 
the transform engine of claim 1 , but doesn't teach conversion from one format. 
Xiong teaches the capability to convert the panoramic data from a first format to 
one or more formats and/or vice versa (i.e. conversion of 3D rays to 2D image, 
see Figure 2 and also column 4, lines 34 - 45). It would have been obvious to 
one of the ordinary skill in the art at the time the invention was made to have 
included the panoramic data conversion as taught by Xiong to the transform 
engine of Henley because it allows an arbitrarily large field of view (see column 
4, lines 54 - 56 of Xiong). 

Regarding claim 12, Xiong teaches that the panoramic data conversion is 
between different coordinates (an ordinary rectilinear lens is projection from a 3D 
ray to a 2D image position; see column 3, lines'! 2-24, column 4, lines 34-67, and 
column 6, line 1 - column 7 line 26). 

Regarding claim 13, Xiong provides a plurality of images, each of the 
plurality of images including a plurality of parameters and optimizing the plurality 
of parameters for each of the plurality of images in accordance with a set of 
distortion functions that is based on a set of polynomials of suitable degree 
(i.e. r = c,0 + c 2 02 + c 3 0 3 + , where the order of polynomial can be determined 

experimentally, see column 4, lines 34-67). 

Regarding claims 34-36, these claims are method claims and analyzed as 
previously discussed with respect to the apparatus claims 11-13 above. 
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9. Claims 17 and 40 are rejected under 35 U.S.C. § 103(a) as being unpatentable 

over Henley, US 5,657,073 in view of Smith, US 6,226,658. 

Regarding claims 17 and 40, Smith teaches a layout code tuning in 
universally readable document files. More specifically, as disclosed in column 2, 
lines 28 - 46, Smith discloses a system and method in which page layout code in 
so-called "universal" document file formats is customized to make the file format 
more universally compatible across a spectrum of different manufacturers. It 
would have been obvious to one of the ordinary skill in the art at the time the 
invention was made to have included the universally readable document files as 
taught by Smith to the imaging system of Henley because this feature makes it 
possible for arbitrary types and numbers of elements to be managed by universal 
file format. 

Conclusion 

10. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. 

- Iwaki et al. (US 2003/01 97780) discloses an omnidirectional camera to 
provide image data having image conversion parameters written in an 
unphotographed area of an annular image shot by a PAL camera lens. 

- Greguss (US 4,566,763) discloses a method and apparatus for a panoramic 
imaging block for three-dimensional space being suitable for pictorial 
recording and displaying of three-dimensional spaces based on the flat 
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cylindrical perspective and having reflecting and refracting surfaces to be 
described mathematical functions. 
- Zimmermann (US 5,185,667) discloses a device for omnidirectional image 
viewing providing pan-and-tilt orientation, rotation, and magnification within a 
hemispherical field-of-view which produces a circular image of an entire 
hemispherical field-of-view, which can be mathematically corrected using high 
speed electronic circuitry. 

Inquiries 

1 1 . Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Kent Wang whose telephone number is 571-270-1703. 
The examiner can normally be reached on 8:00 A.M. - 5:30 PM (every other Friday off). 
If attempts to reach the examiner by telephone are unsuccessful, the 
examiner's supervisor, Chanh Nguyen can be reached on 571-272-7772. The 
fax phone number for the organization where this application or proceeding is 
assigned is 571-270-8300. 

Information regarding the status of an application may be obtained from 
the Patent Application Information Retrieval (PAIR) system. Status information 
for published applications may be obtained from either Private PAIR or Public 
PAIR. Status information for unpublished applications is available through 
Private PAIR only. For more information about the PAIR system, see http://pair- 
direct.uspto.gov. Should you have questions on access to the Private PAIR 
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system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll- 
free). If you would like assistance from a USPTO Customer Service 
Representative or access to the automated information system, call 800-786- 
9199 (IN USA OR CANADA) or 571-272-1000. 



Kent Wang 

12 February 2007 
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